gh-121103: Put free-threaded libraries in `lib/python3.14t` by colesbury · Pull Request #121293 · python/cpython

colesbury · 2024-07-02T20:22:16Z

On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation.

Issue: Free-threaded libraries should be in lib/python3.14t (configure) #121103

On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation.

colesbury · 2024-07-02T20:43:48Z

pip, uv, venv, setuptools, seem to work, but I don't entirely know what I'm doing, so I'd appreciate a thorough review.

This also fixes the test_sysconfig failure on Windows.

hroncok · 2024-07-03T09:39:43Z

Does this mean that all of the pure Python standard library files would need to be installed in lib/python3.14t
as well? I.e. when I have both the "normal" and the freethreaded Python installed, do I need all the standard library modules twice?

encukou · 2024-07-03T12:32:12Z

I don't have very deep knowledge in this area, so I can't easily do a thorough review :(
I'd trust @FFY00 and @hroncok's reviews if they can contribute their time for it.

erlend-aasland · 2024-07-03T12:41:05Z

Does this mean that all of the pure Python standard library files would need to be installed in lib/python3.14t as well? I.e. when I have both the "normal" and the freethreaded Python installed, do I need all the standard library modules twice?

As far as I can tell, from a quick look at this PR: yes.

hroncok · 2024-07-03T13:01:19Z

As far as I can tell, from a quick look at this PR: yes.

I thought so.

The real question is: Is this the outcome we anticipated?

If not, I suggest keeping the stdlib and platstdlib values intact.

colesbury · 2024-07-03T14:41:32Z

Yes, that was the intention. The goal is to separate the installation. We run into a lot of potential problems if we have partially overlapping stdlib files.

erlend-aasland · 2024-07-03T16:26:16Z

Yes, that was the intention. The goal is to separate the installation. We run into a lot of potential problems if we have partially overlapping stdlib files.

FTR, I totally agree with this.

colesbury · 2024-07-03T16:31:23Z

To expand on my previous comment: if we have kept a common stdlib and platstdlib, the Python files would be shared, but the .so files would not. So I'd be concerned that if someone installed, e.g. 3.13.0 default and 3.13.8 free-threaded to the same prefix, then the .py files and native .so libraries might not match. Keeping things separate in lib/python3.13t vs. lib/python3.13 avoids this problem.

colesbury · 2024-07-10T15:05:42Z

I plan to merge this tomorrow if there are no objections

ned-deily

Other than the nit comment, +1. This looks like it does what it promises. We still have a bit more to do before a free-threaded build can be installed to the same prefix as a non-free-threaded build without conflicts, primarily due to duplicate symlinked names in the bin and lib/pkgconfig directories. But this isn't really a new problem as there are similar conflicts when sharing with a debug build prior to this PR. I'd say let's merge this PR now and we can follow up in another PR for which I want to do some more investigation.

Makefile.pre.in

miss-islington-app · 2024-07-11T20:21:40Z

Thanks @colesbury for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

…thonGH-121293) On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation. (cherry picked from commit e8c91d9) Co-authored-by: Sam Gross <colesbury@gmail.com>

bedevere-app · 2024-07-11T20:21:52Z

GH-121631 is a backport of this pull request to the 3.13 branch.

…H-121293) (#121631) On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation. (cherry picked from commit e8c91d9) Co-authored-by: Sam Gross <colesbury@gmail.com>

…thon#121293) On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation.

AlexWaygood · 2024-08-06T11:16:03Z

Lib/site.py

+        if hasattr(sys, 'abiflags') and 't' in sys.abiflags:
+            abi_thread = 't'
+        else:
+            abi_thread = ''
        if os.sep == '/':
            libdirs = [sys.platlibdir]
            if sys.platlibdir != "lib":
                libdirs.append("lib")

            for libdir in libdirs:
                path = os.path.join(prefix, libdir,
-                                    f"{implementation}{ver[0]}.{ver[1]}",
+                                    f"{implementation}{ver[0]}.{ver[1]}{abi_thread}",


@colesbury -- would you be able to update the docs for the site module to reflect these changes to how Python finds the site-packages directory?

(The context is I'm currently working on reimplementing Python's logic here for Astral's work-in-progress type checker. I was reading through the site.py source code, and it surprised me to find logic here that wasn't documented!)

Thanks for the pointer. I put up a PR to update the docs: #122737

pythongh-121103: Put free-threaded libraries in lib/python3.14t

03772aa

On POSIX systems, excluding macOS framework installs, the lib directory for the free-threaded build now includes a "t" suffix to avoid conflicts with a co-located default build installation.

colesbury added topic-free-threading needs backport to 3.13 bugs and security fixes labels Jul 2, 2024

bedevere-app bot mentioned this pull request Jul 2, 2024

Free-threaded libraries should be in lib/python3.14t (configure) #121103

Closed

Fix blurb

ed2dde7

colesbury marked this pull request as ready for review July 2, 2024 20:42

colesbury requested review from FFY00, corona10, erlend-aasland and vsajip as code owners July 2, 2024 20:42

bedevere-app bot added the awaiting core review label Jul 2, 2024

colesbury requested review from Yhg1s, encukou and ned-deily July 2, 2024 20:42

Make ABI logic in sysconfig a bit more consistent with everything else.

6b46646

This also fixes the test_sysconfig failure on Windows.

corona10 approved these changes Jul 3, 2024

View reviewed changes

bedevere-app bot added awaiting merge and removed awaiting core review labels Jul 3, 2024

ned-deily approved these changes Jul 11, 2024

View reviewed changes

Makefile.pre.in Outdated Show resolved Hide resolved

colesbury added 2 commits July 11, 2024 14:22

Use tab in Makefile.pre.in

d72fb70

Merge branch 'main' into pythongh-121103-libdir

432701c

erlend-aasland approved these changes Jul 11, 2024

View reviewed changes

colesbury merged commit e8c91d9 into python:main Jul 11, 2024

bedevere-app bot removed the awaiting merge label Jul 11, 2024

colesbury deleted the gh-121103-libdir branch July 11, 2024 20:21

bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Jul 11, 2024

colesbury mentioned this pull request Jul 13, 2024

Having two identical executables for free-threaded version is misleading pyenv/pyenv#3015

Open

10 tasks

AlexWaygood reviewed Aug 6, 2024

View reviewed changes

FFY00 mentioned this pull request Dec 4, 2025

gh-131372: Include LDVERSION and EXE in base_interpreter value #142256

Merged

Uh oh!

Conversation

colesbury commented Jul 2, 2024 • edited by bedevere-app bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

colesbury commented Jul 2, 2024

Uh oh!

hroncok commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

encukou commented Jul 3, 2024

Uh oh!

erlend-aasland commented Jul 3, 2024

Uh oh!

hroncok commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

colesbury commented Jul 3, 2024

Uh oh!

erlend-aasland commented Jul 3, 2024

Uh oh!

colesbury commented Jul 3, 2024

Uh oh!

colesbury commented Jul 10, 2024

Uh oh!

ned-deily left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app bot commented Jul 11, 2024

Uh oh!

bedevere-app bot commented Jul 11, 2024

Uh oh!

AlexWaygood Aug 6, 2024

Choose a reason for hiding this comment

Uh oh!

colesbury Aug 6, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

colesbury commented Jul 2, 2024 •

edited by bedevere-app bot

Loading

hroncok commented Jul 3, 2024 •

edited

Loading

hroncok commented Jul 3, 2024 •

edited

Loading